Flexible Turn-Taking for Spoken Dialogue Systems

Most of the research on spoken dialogue systems so far has focused either on higher levels of dialogue or on speech understanding. In contrast, the lowlevel interactional aspects of conversation such as turn-taking have been essentially ignored, leading builders of practical systems to resort to simple pause detection-based methods to handle turn-taking. In a preliminary study based on the Let’s Go bus information system, I found that such methods lead to interaction failures and potentially to complete dialogue breakdowns for a significant proportion of the dialogues with real-world users. In addition, a comparison of conversational rhythm in successful dialogues showed that even when speech recognition is not a major obstacle to communication, systems perform very differently and much less efficiently than human speakers. To address these issues, I propose an approach that relies on two innovations over current dialogue systems. First, it features a new event-driven system architecture that allows real-time processing of conversation, which I implemented in the Olympus/RavenClaw spoken dialogue framework. In addition to the dialogue manager and the traditional understanding and generation modules, a new module, the interaction manager, is in charge of dynamically monitoring and managing low-level interaction phenomena. The second component of the proposed approach is the turn-taking model used by the interaction manager. Inspired by mobile robotics and autonomous agent research, the model is composed of a set of sensors that provide information about the world, a set of actions that the system can take, and an action selection mechanism. Although I will explore different such mechanisms, Reinforcement Learning appears to be an appropriate framework for learning turn-taking behavior. The last expected contribution of this thesis is in the form of an evaluation framework for turn-taking in spoken dialogue systems. This important aspect of the proposed work will include a study of various local and global metrics of turn-taking and dialogue, along with the design and validation of composite metrics. All the theoretical findings and models proposed in this thesis will be grounded and validated in real world applications including Let’s Go and other RavenClaw-based dialogue systems.